REMARKS 



In the Office Action dated June 25, 2008, claims 1-12 were presented for examination. 
Claims 3-5 and 8-10 were rejected under 35 U.S.C. §101 as being directed to non-statutory 
subject matter. The Specification and Drawings were objected to as containing informalities. 
Claims 1 and 3-7 were rejected under 35 U.S.C. § 102(a) as anticipated by Nilsson, id3v2 
Informal Standard. Claims 2, 8, 9, 11 and 12 were rejected under 35 U.S.C. § 103(a) as being 
unpatentable over Nilsson, in view of Rio, Rio PMP 300 User's Guide. 

In this Amendment, Applicant has canceled claims 3 and 8 from further consideration in 
this application. Applicant is not conceding that the subject matter encompassed by claims 3 and 
8 is not patentable. Claims 3 and 8 were canceled in this Amendment solely to facilitate 
expeditious prosecution of the remaining claims. Applicant respectfully reserve the right to 
pursue additional claims, including the subject matter encompassed by claims 3 and 8, as 
presented prior to this Amendment in one or more continuing applications. 

I. Objection To Specification 

In the Office Action dated June 25, 2008, the Examiner objected to the content of the 
Specification. More specifically, the Examiner identified the inconsistencies of the titles in the 
originally submitted application. Applicant has attached a substitute specification. No new 
matter has been added to the pending application by way of the Substitute Specification. Since 
there are numerous changes, Applicant respectfully requests that the Examiner merely replace the 
original specification with the enclosed substitute specification. 

II. Objection To Drawings 

In the Office Action dated June 25, 2008, the Examiner objected to the drawing figures. 
However, the Examiner did not identify any specific elements in the drawing figures that require 
correction. Accordingly, Applicant respectfully request that the Examiner either remove the 
objection or clearly identify the items that require correction. 
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III. Rejection Under 35 U.S.C. §101 

In the Office Action dated June 25, 2008, the Examiner rejected claims 3-5 and 8-10 
under 35 U.S.C. §101 as being directed to non-statutory subject matter. Applicant has cancelled 
claim 3. As such, the rejection of claim 3 is now moot. 

With respect to claims 4 and 5, the Examiner has indicated that the term "the caption 
MP3 data format" may be interpreted as falling within a non-statutory category of invention. 
Applicant has amended claims 4 and 5 by substituting "caption MP3 data format" with "caption 
MP3 player" which clearly falls under definition of statutory subject matter. In addition, claims 4 
and 5, initially dependent upon claim 3, have been made dependent upon claim 1 reciting 
statutory subject matter. Support for this amendment is found on page 3, lines 3-20 of 
Applicant's specification. No new matter has been added. 

With respect to claims 9 and 10, the Examiner has indicated that the claimed subject 
matter appears to be non-functional descriptive material stored on computer readable medium. 
To overcome the above rejection, Applicant has amended claim 9, initially dependent upon claim 
8, by making it dependent upon claim 6 reciting statutory subject matter. Claim 10 falls within a 
statutory category as being dependent upon newly amended claim 9. Accordingly, no new matter 
has been added with the amendment presented herein. 

It is Applicant's position that the subject matter of claims 4, 5, 9, and 10 as amended 
constitutes statutory subject matter. Accordingly, Applicant respectfully request entry of the 
amendment to claims 4, 5, 9, and 10 and removal of the rejection under 35 U.S.C. §101. 

IV. Rejection to claims 1 and 3-7 under 35 U.S.C. § 102(a) 

In the Office Action dated June 25, 2008, the Examiner rejected claims 1 and 3-7 under 
35 U.S.C. § 102(a) as being anticipated by Nilsson. 

Applicant's invention pertains to an MP3 player reproducing caption information. More 
specifically, an MP3 file comprises audio and caption information. The audio information within 
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a standard MP3 is divided into a plurality of audio frame units. The caption information 
includes position and /or time data providing correspondence between audio frame units and 
caption information. The control means of the MP3 player synchronizes the audio data frame 
units with the caption information using position and time data. As result, audio and caption data 
is output synchronously while the player is playing. See page 3, lines 3-14 of Applicant's 
specification. Accordingly, each audio frame unit within one MP3 file has its own caption 
information. 

In contrast, Nilsson teaches an ID3v2 tag consisting of a number of frames, each of 
which contains caption information. According to Nilsson, the caption information is directed to 
the audio information contained in the file as a whole. Accordingly, Nilsson does not disclose 
dividing audio data within one MP3 file in a plurality of frame units, such that each frame unit 
has its own caption. As result, there is no teaching in Nilsson in relation to synchronize the audio 
frame units with the caption information using position and/or time data that is a part of caption 
information. Applicant has amended claim 1 to distinguish his invention over Nilsson. The 
amendment is aimed at distinguishing the limitations discussed above from the prior art as 
applied by the Examiner. Support for the amendment is found on page 3, lines 3-20 of 
Applicant's specification. No new matter has been added with the amendment presented herein. 

Under the law of anticipation, "[f]or a prior art reference to anticipate in terms of 35 
U.S.C. §102, every element of the claimed invention must be identically shown in a single 
reference." 1 It is respectfully submitted that in view of the amendment presented herewith, 
Nilsson does not expressly or inherently teach or show every element as presented in Applicant's 
claims 1 and 3-7. Accordingly, Applicant respectfully request that the Examiner remove the 
rejection under 35 U.S.C. §102(a) and request allowance of claims 1 and 3-7. 

V. Rejection to claims 2, 8, 9, 11 and 12 under 35 U.S.C. §103(a) 

In the Office Action dated June 25, 2008, the Examiner rejected claims 2, 8,9, 11 and 
12 under 35 U.S.C. § 103(a) as being unpatentable over Nilsson in view of Rio. 

1 Diversitech Corp. v. Century Steps, Inc., 7 USPQ2d 1315, 1317 (Fed. Cir. 1988). 
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Applicant's remarks pertaining to Nilsson made above are hereby incorporated by 
reference. 

With respect to claims 1 1 and 12, Applicant teaches an MP3 file divided into a plurality 
of MP3 audio frame units, each of which comprises audio information and corresponding caption 
information. As noted above, Nilsson does not teach data format allowing to divide an MP3 file 
into multiple audio frame units containing both audio and caption information. 

The Examiner employs Rio as a prior art reference in relation to a liquid crystal display 
serving as a caption output means. In addition, Rio is cited in relation to caption display time data 
being compared with a play time counted when the MP3 file is reproduced. 

Rio discloses a Rio PMP 300 player featuring a LED display. The LED display 
indicates current mode (Play, Pause, Stop), status, and additional information including volume 
level, track number/time counter, and battery charge status. See page 12 of Rio. However, the 
above information can not be considered as caption information which is defined as an 
explanatory comment or designation accompanying audio data. In addition, Rio does not 
disclose comparing caption display time data with a play time counted when the MP3 file is 
reproduced. 

To support the rejection set forth, claims 2 and 9 each require the limitation of the 
associated independent claims. If the prior are references do not teach or suggest every claim 
limitation of the Applicant's invention, then they do not meet every requirement under 35 U.S.C. 
§ 103(a) and are not sufficient to uphold a rejection under 35 U.S.C. § 103(a). 2 In the present 
case, as stated above, Applicant teaches a method involving dividing audio information within 
one MP3 file in a plurality of frame units and synchronizing the audio information with caption 
information such that each audio frame unit has its own caption data assigned. Therefore, 
because neither Nilsson or Rio teach the above limitations claimed by Applicant, the combination 
of references as applied by the Examiner does not teach every element of Applicant' s claimed 

2 See MPEP §2143. 
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invention. The Nilsson reference is not sufficient to uphold a rejection under 35 U.S.C. § 103(a), 
and as such, the secondary references, which also lack the cited elements, in combination are also 
insufficient to uphold the rejection under 35 U.S.C. § 103(a). Accordingly, Applicant respectfully 
requests that the Examiner remove the rejection and direct allowance of claims 2, 9, 11 and 12. 

VI. Rejection to claim 10 under 35 U.S.C. §103(a) 

In the Office Action dated June 25, 2008, the Examiner rejected claim 10 under 35 
U.S.C. § 103(a) as being unpatentable over Nilsson in view of Rio, and further in view of Matz et 
al, U.S. Patent No. 6,198,511. 

Applicant's remarks pertaining to Nilsson and Rio made above are hereby incorporated 
by reference. 

The Examiner employs Matz et al. in relation to a caption including additional data such 
as a scroll type and flash information. 

Matz et al. teaches an interactive broadcast including television programming and 
associated services such as closed caption scripts. More specifically, the audio portion of the 
broadcast television programming, in closed caption script format, is compared to a key word 
database. The closed caption text may be displayed in real time. When a match is identified, an 
event is fired. 

Claim 10 is dependent upon claim 9, which, in its turn, is dependent upon claim 6. If the 
prior art references do not teach or suggest every claim limitation of the Applicant's invention, 
then they do not meet every requirement under 35 U.S.C. § 103(a) and are not sufficient to uphold 
a rejection under 35 U.S.C. § 103(a). 3 As was noted above, the Nilsson reference is not sufficient 
to uphold a rejection of claim 6 under 35 U.S.C. § 102(a), and as such, the secondary references 
in combination are also insufficient to uphold the rejection under 35 U.S.C. § 103(a). 



See MPEP §2143. 
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Accordingly, Applicant respectfully requests that the Examiner remove the rejection and direct 
allowance of claim 10. 

VII. Conclusion 

In view of the forgoing amendment to the claims, it is submitted that all of the claims 
remaining in the application are now in condition for allowance and such action is respectfully 
requested. Applicant is not conceding in this application that those claims in their prior forms 
are not patentable over the art cited by the Examiner, as the present claim amendments and 
cancellations are only for facilitating expeditious prosecution of the application. Applicant 
respectfully reserves the right to pursue these and other claims in one or more continuation 
and/or divisional patent applications. Should any questions arise in connection with this 
application or should the Examiner believe that a telephone conference with the undersigned 
would be helpful in resolving any remaining issues pertaining to this application, the undersigned 
respectfully requests that she be contacted at the number indicated below. 

For the reasons outlined above, withdrawal of the rejection of record and an allowance 
of this application are respectfully requested. 



Respectfully submitted, 
By: /Rochelle Lieberman / 
Rochelle Lieberman 
Registration No. 39,276 
Attorney for Applicant 

Lieberman & Brandsdorfer, LLC 

802 Still Creek Lane 

Gaithersburg, MD 20878 

Phone: (301) 948-7775 

Fax: (301) 948-7774 

Email: rocky@legalplanner.com 

Date: December 22, 2008 
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Attorney Docket No. 1298-201 

Substitute Specification (With Markings) 

CAPTION MP3 PLAYER HAVING A CAPTION DISPLAY FUNCTION, 
CAPTION MP3 DATA FORMAT AND METHOD OF REPRODUCING 
CAPTION MP3 DATA 

5 CROSS REFERENCE TO RELATED APPLICATION(S) 

This application claims priority to international application number 
WO/2000/41 175 filed under the patent cooperation treaty, which claims priority to 
Korean Patent Application No. 199/235, filed on January 8, 1999, which are hereby 
incorporated herein by reference. 

10 BACKGROUND OF THE INVENTION 

[Field of the Invention] 

The present invention relates to a caption MP3 player having a caption display 
function, caption MP3 data format and method of reproducing caption MP3 data and in 
particular, to a caption MP3 player having a function of displaying caption information on 
15 a display device in synchronism with corresponding audio information while outputting 

the audio information on stereo, a caption MP3 data format and a method of reproducing 
the caption MP3 data. 

[Description of the Related Art] 

Generally, MP3 means MPEG (Motion Picture Expert Group) Layer-3, and 
20 belongs to the audio technology in the MPEG field. MP3 is an audio file format that is 

formed by compressing existing data by audio data coding without deterioration in sound 
quality. Such an MP3 file has an AAU (hereinafter, "audio decoding unit") recording 
format. In other words, the MP3 file comprises a header, cyclic redundancy check (CRQ, 
audio information, and auxiliary data. Usually, the MP3 player playing MP3 files is used 
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as a dedicated audio appliance for receiving compressed audio files and reproducing them 
in the form of audio information. 

The conventional MP3 player mounts a liquid crystal display on which, in 
addition to the audio information, simple character data (e.g., a simple reference such as a 
5 title of a song) is displayed. However, such character data cannot synchronously display a 

character, i.e., the caption information, which corresponds to the audio information. 

In the conventional caption tape type using a cassette tape, the caption information 
and the audio information are stored on two tracks of the tape, respectively, that is used 
exclusively for audio. The audio information and the caption information are outputted to 
10 the speaker and liquid crystal display, respectively, by the caption cassette player. 

However, the caption tape type, in which the digital signal of the character data is 
converted into an analog signal and stored on the tape, has some problems when 
reproducing the data. The problems are that: the character signal results in noises by 
interrupting the audio signal; the audio signal results in an error in the character by 
15 interrupting the character signal; or the audio information is outputted on mono, not 

stereo, by storing the character data in one of the tracks on the tape. 

To solve the problem of mono output in the caption cassette, the tape is divided 
into four tracks or output is achieved in the stereophonic mode by signal synthesis. 
However, when four tracks are used, the player should comprise a four-track head to 
20 process the data on each track. When the data is outputted in the stereophonic mode by 

the signal synthesis, signal loss may occur during the analysis of the signal since it is not 
possible to completely divide the synthesized signals. Moreover, both signals result in 
noises by interrupting each other when reproducing the audio information 

SUMMARY OF THE INVENTION 
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An object of the present invention is to provide an MP3 player having a caption 
information display function of storing audio information and corresponding caption 
information in an MP3 recording medium and reproducing the recorded data by 
synchronizing the data with each other, and provide an MP3 data format and a method of 
5 reproducing the MP3 data. 

An MP3 file according to the present application comprises standard MP3 audio 
information and caption information having data to display the audio information in the 
form of characters, and thus is referred to hereinafter as caption MP3 data or caption W3 
file. 

10 In the reproduction of the caption MP3 file, the audio information to be 

reproduced and the corresponding caption information should be synchronously 
outputted. For the synchronization of the audio information and the caption information, 
position data and time data can be used. Both or only one of the position data and the 
time data can be used. The position data may be that of the audio information that should 

15 be synchronized with the caption information or that of the caption information that 

should be synchronized with the audio information. The time data is to indicate the 
display time of the caption data that should be outputted through a display device. 

The caption MP3 player according to the present invention reproduces the caption 
MP3 data comprising audio information and corresponding caption information (the 

20 caption information including position data and/or time data), the audio information 

having a standard MP3 file format comprising a header, audio data and auxiliary data, the 
caption MP3 player comprising: a storage means for storing the audio information and the 
corresponding caption information inputted thereto; a signal separating means for 
separating the audio information and caption information inputted from the storage 

25 means; a control means for controlling the storage and output of the information through 

the storage means and controlling the audio signal and the corresponding caption signal, 
which are separated by the signal separating means, to be synchronized; a caption output 
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means for outputting the caption signal synchronized with the audio signal, which 
corresponds to the caption signal and is outputted from the audio output means, by 
inputting the output from the signal separating means, The caption NW3 data format 
according to a first embodiment of the present invention comprises an audio information 
5 and a corresponding caption information, the audio information having a standard NW3 

file format with a header, audio data and auxiliary data, wherein the caption information 
includes position data and/or time data, and when the audio information is reproduced, 
the caption information synchronized with the reproduced audio information is outputted 
using the position data and/or the time data. 

10 The caption MP3 data format according to a second embodiment of the present 

invention has a format comprising a plurality of caption MP3 files, each of the caption 
MP3 files having audio information and corresponding caption information, the audio 
information having a standard XT3 file format with a header, audio data and auxiliary 
data; wherein the audio information is located before the caption information in the each 

15 MP3 file, and the caption information includes caption display time data showing the 

time indicated in a display device when the caption information is reproduced. 

According to the present invention, the caption information can be included in the 
standard MP3 information, in which the audio information is stored, and the caption 
information can be provided along with the audio information by synchronously 
20 outputting the caption information with the audio information using the position data 

and/or time data. 

The present invention has an advantage in that there is no noise by the 
intervention between the caption information and the audio information since they are 
separated. The present invention has an advantage in that the caption information is 
25 stored in the form of a digital signal so that the caption information can be stored in 

various formats such as an image, hypertexter, text, etc. and that the deterioration in 
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sound quality by repeated reproduction is prevented. 

Further, there is an advantage in that the file format, in which the audio and the 
caption are synchronized, makes searching and movement between intervals faster and 
easier. 

5 BRIEF DESCRIPTION OF THE DRAWINGS 

The drawings referenced herein form a part of the specification. Features shown 
in the drawing are meant as illustrative of only some embodiments of the invention, and 
not of all embodiments of the invention unless otherwise explicitly indicated. 
10 Implications to the contrary are otherwise not to be made. 

FIG. I is a block diagram of a caption NW3 player format according to the present 
invention. 

FIG. 2 is a view illustrating a caption XT3 data format according to the first 
embodiment of the present invention. 

15 

FIG. 3 is a flow chart showing a caption XW3 data reproduction method 
according to the first embodiment of the present invention. 

FIG. 4 is a view illustrating the caption NP3 data format according to the second 
embodiment of the present invention. 

20 

FIG. 5 is a flow chart showing the method of reproducing caption information in 
the caption NW3 data according to the second embodiment of the present invention. 



Patent Application Specification 



5 



Attorney Docket No. 1298-201 



DETAILED DESCRIPTION OF THE INVENTION 

The construction and operation of the present invention will be explained in detail 
with reference to the accompanying drawings. 

FIG. 1 is a block diagram of an MP3 player having a caption display function 
5 according to the present invention. The caption MP player according to the present 

invention comprises an input section (11), storage section (12) for storing audio 
information and/or caption information, signal separation section (13) for separating the 
audio information and the caption information, control section (10), audio output section 
(16), caption output section (14) and display section (15). 

10 The audio information and the caption information are inputted through the input 

section (11) from a recording medium (18) in which caption MP3 files are recorded. The 
input section (1 1) is preferably an electric circuit comprising a connection port. The audio 
information and the caption information inputted through the input section (1 1) are stored 
in the storage section (12), preferably a memory cell. The signal separation section (13) 

15 separates the audio information and the caption information stored in the storage section 

(12). 

The control section (10) controls the storage of information in the storage section 
(12) or the output of information from the storage section (12), and synchronizes audio 
signals and caption signals that are separated in the signal separation section (13). Also, 
20 the control section (10) counts playing time while the audio information is being 

reproduced. 

The control section (10) is preferably a microcomputer having function of 
processing signals or a control circuit comprising a microcomputer, and is formed to be 
controlled by a user. 
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The audio output section (16) receives the audio signal corresponding to the audio 
information among the information from the signal separation section (13) and sends the 
signal to a left speaker (L SPK) and a right speaker (R SPK) so that the signal is outputted 
as an audio signal that can be heard. The caption output section (14) outputs the caption 
5 signal corresponding to the caption information among the information from the signal 

separation section (13) in synchronism with the audio signal outputted from the audio 
output section (16). 

The display section (15) displays the caption corresponding to the caption signal 
10 outputted from the caption output section (14) in a visible form on the screen. Preferably, 

the display section is a liquid crystal display (LCD). 

In the caption MP player of the present invention having such a construction as 
described above, when the audio information and the caption information are inputted 
through the input section (11) from the caption MP3 file recorded in the recording 

15 medium (18), the audio information and the caption information are stored in the storage 

section (12) through the control of the control section (10). Each information stored in the 
storage section (12) is outputted from the storage section (12) through the control section 
(10) controlled by the user when the output of information is required. The outputted 
information is separated into audio information and caption information through the 

20 signal separation section (13), and the separated audio signal is outputted to the speaker 

in the monophonic or stereophonic mode through the audio output section (16). The 
separated caption signal is synchronized with the audio signal and outputted on the 
display section (15) through the caption output section (14). 

First Embodiment 

25 A caption MP3 data format according to the first embodiment of the present 

invention will be described. 
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FIG. 2 is a view showing a caption MP3 data format according to an embodiment 
of the present invention. The caption MP3 information according to the present invention 
consists of audio information (20) and caption information (22). The audio information 
has a standard NT3 file format with a header, CRC, audio data and auxiliary data. The 
5 header is located on a fixed field of 32-bits and in that field, information, such as a layer, 

sampling frequency and remaining frame, is contained. As an option, the existence or 
nonexistence of CRC depends on the header. Audio data is compressed data, and the 
length of data depends on the kind of the data. The auxiliary data, as a user definition 
area, includes additional information and is variable in dimension. 

10 Each caption information (22) comprises a start synchronization code (1), 

reproduction number data (2) and information data (8). It is not necessary to arrange these 
elements in the order as shown in FIG. 2. The arrangement shown in FIG. 2 is just an 
example for description. 

The start synchronization code (1) of the caption information (22) means the 
15 beginning of the caption information. The reproduction number data (2) is located after 

the start synchronization code (1) and indicates the number for indicating to which audio 
information frame among a plurality of pieces of audio information (20) the caption 
information corresponds. It can be understood that the reproduction number is the 
position data of the audio information with which the caption information should be 
20 synchronized or the position data of the caption information with which the audio data 

should be synchronized. 

The reproduction number data indicates the number, which is used for reference 
when the audio information (20) and the caption information (22) are reproduced, and is 
formed with a size of 4-bits, for example. 

25 The information data (8) include the related information such as the address of 

data or the kind of data to be stored and also includes, for example, reproduction address 
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data (3), information identification code (4), selection code (5) and caption data (6). 



The reproduction address data (3) shows the reproduction number, by which the 
caption information is combined with each other when a plurality of pieces of caption 
information are in the form of one word or picture, and is formed with a size of 4-bits, for 
5 example. To form a paragraph of large amount, at least one or more pieces of the caption 

information are required. The information identification code (4) shows of what type a 
stored information file is. The file can be in the form of an image file, hypertext file 
(HTML) or text file, for example, which is adapted for the display device. 



The selection code (5) indicates at least one of the language form used in the 
10 stored information, operation time and display mode of the display section (15). The form 

of the used language can be, for example, Korean (KOR), Japanese QP), English (USA), 
etc., and the operation time is the time at which the caption information should be 
outputted. By using the operation time, the caption information is synchronized with the 
audio information. 



15 The display mode shows whether the caption in the form of a word or sentence is 

outputted on the display section (15) in sequence or at once, and determines in what form 
(for example, columns and 4 lines or 24 columns and 2 lines) the characters should be 
displayed. 

By using the operation time of the selection code (5), it is possible to output the 
20 caption information in synchronism with the audio information. For such 

synchronization, the operation time data and the reproduction number data (2) can be 
used together or separately. 



The caption data (6) shows the caption character outputted from the MP3 
recording medium. The character stored at this time can be, for example, in the form of an 
5 image, hypertext, text, etc. When a caption information group (22a) comprising a 
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plurality of pieces of caption information (22) formed as such is added to an W3 data 
format (20a) comprising several pieces of audio information (20) of an audio decoding 
unit and is outputted from the recording medium in which the MP3 audio information 
(20) is stored, the caption character is outputted in synchronism with each audio signal 
5 extracted from a plurality of pieces of audio information (20). The MP3 data group (20a) 

forms one MP3 file that for example, corresponds to a song. The caption data group (22a) 
includes the contents of caption corresponding to one MP3 data format (20a), and each of 
a plurality of pieces of caption information (22) corresponds to one word or sentence that 
forms the contents of the caption. 

10 

The recording medium comprises an optical recording apparatus such as a 
compact disc, audio tape, magnetic recording apparatus, such as a hard disc, and memory. 

A plurality of pieces of audio information (20) included in the MP3 audio 
information (20a) each has a 32-bit header and 16-bit CRC, audio data and additional 
15 data. 

Reproduction of the information having such a caption MP3 data format will be 
explained with reference to the flow chart of FIG. 3. 

The recording medium is inserted into an apparatus reproducing the signal stored 
20 in the recording medium, e.g., the MP3 player shown in FIG. 1, and the stored 

information is reproduced by the control of the control circuit including a microcomputer 
of the reproducing apparatus. 

At least one of the audio information (20) and the caption information (22) is 
stored in the MP3 recording medium (step 30) so that the synchronized audio signal and 
25 the caption signal are outputted for reproduction. The control circuit determines whether 

the audio information (20) only exists in the information stored in the recording medium 
(step 32). 
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At the determining step (32), when it is determined that the audio information 
(20) exists in the recording medium without the caption information (22), the audio 
information (20) only is outputted from the recording medium (step 34) because the 
caption information (22) is not outputted and thus does not exist. 

5 However, when it is determined that the caption information (22) exists in the 

recording medium, the caption signal and the audio signal are reproduced in synchronism 
with the caption information according to the existence and/or non- existence of the audio 
information (20) in the recording medium, and then the reproduction of the next caption 
signal or audio signal is repeated (steps 36-48). 

10 As a first determining step, to reproduce all of the desired caption information and 

the audio information during the above repetition, it should be determined first whether 
the caption information (22) and the audio information (20) exist together in the 
recording medium (step 36). 

At the first step (36), if it is determined that the caption information (22) exists 
15 without the audio information (20), the caption information (22) only is outputted (step 

38). In the meantime, the caption information (22) exists in the first step (36) together 
with the audio information (22), the compressed audio information (20) is decoded for 
reproduction (step 40). 

As a second determining step, it is determined whether the caption information 
20 (22) corresponding to the audio information (20), which is decoded at the decoding step 

(40) so that when the reproduction apparatus outputs an audio, a corresponding caption 
can be outputted (step 42). If there is no corresponding caption information, output of 
information is maintained by continuously outputting the already outputted caption 
information or a caption information having a blank character (step 44). 
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However, if there is caption information (22) corresponding to the audio 
information (20), the caption information is decoded according to the file form of the 
caption information, for example, image, hypertext, text, etc (step 46). 

Then, the caption information is synchronized with the corresponding audio 
5 information, which is decoded at the decoding step (40), in the reproduction apparatus 

and outputted (step 48), and the first determining step (36) returns to output the next 
caption information. 

For example, by the caption information (22) added to the audio information (20), 
the sound "beau-" and the character "beau-" synchronized with the sound "beau-" are 
outputted from the respective output apparatuses, i.e., the speaker from which the audio 
information (20) is outputted and the display device from which the caption information 
(22) is concurrently outputted. At this time, since the audio information (20) and the 
caption information (22) have a capacity capable of storing at least a part of the audio 
signal and the caption signal, which are to be "beau-," the audio information (20) and the 
corresponding caption information (22) are required to store words such as "beautiful". 

In this manner, the audio information (20) stored in the caption MP3 recording 
medium and the caption information (22) synchronized to correspond to the audio 
information are simultaneously reproduced as audio and caption through the respective 
output apparatuses. 

20 Second Embodiment 

As shown in FIG. 4, an MP3 data format (50) according to a second embodiment 
of the present invention comprises a plurality of caption MP3 files (50a, 50b,...). Each 
caption MP3 file has audio information (52) and caption information (54). The audio 
information (52) comprises standard MPEG audio files as that in the first embodiment. 
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In FIG. 4, the caption information (54) comes after the audio information (52). In 
a different way, the caption information (54) could come before the audio information 
(52). However, when considering the wide utility of software for MP3 file reproduction, 
it is preferable to use the structure shown in FIG. 4. 

5 In FIG. 4, one MP3 file (e.g., 50a) appears to comprise one audio information 

(52a) and one caption information (54a). However, that is just for simplification of the 
drawing. 

One skilled in the art could understand that substantially in the most caption MP3 
files, one file comprises a plurality of pieces of audio information and a plurality of pieces 
10 of corresponding caption information. One MP3 file corresponds to one paragraph of 

phonetic information that is divided, for example, by a tune of song and a predetermined 
basis (e.g., theme). 

Each of the caption information 1 and 2 (54a and 54b) comprises a caption start 
synchronization signal (56), caption data 1, 2,... N (58a, 58b,..., 58n), text type (67) and 
15 caption identification code (62). The caption start synchronization signal (56), for 

example, as 4-byte data, indicates the start position in which the caption information is 
contained. The caption data (58) contains the character information to be displayed on the 
real screen and its size changes according to the character information. The caption data 
(58) will be explained hereinafter. 

20 Text type (60) determines the type, i.e., form of text output so that for example, 

the character information in the caption data (58) is outputted, being formed into 20 
columns and 4 lines or 24 columns and 2 lines. The caption identification code (62) is a 
code that identifies whether the data format is the caption MP3 file. Each of the caption 
data 1, 2,..., N (58a, 58b,..., 58n) is formed to include a caption display time (64), 

25 sentence start identification (66), caption (68), additional data (70), option (72) and the 
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size of data (74). 

The caption display time (64) as the time data of a point of time to display the 
caption is sized, for example, to be 7 bytes. The sentence start identification (66) is a code 
to find the beginning of the sentence when the sentence is displayed on different screens. 

5 The caption (68) is character data to be displayed on the screen, and the additional 

data (70) is used when an information except the option is required and contains the 
information that shows the form of the caption information file (image file, hypertext file, 
text file) or indicates the language of the caption information. The option (72) is a storage 
place for so-called optional matters that stores the information such as a scroll form (e.g., 
10 the character is displayed, flowing on the screen, or a previous sentence disappears slowly 

and the next sentence appears slowly on the same place) or a flash (e.g., flickering of 
letters). Into the data size (74), information on the length of the caption data is inputted. 

In the caption MP3 file of such an information format, the audio information and 
the caption information are synchronized with each other by the caption display time (64) 
15 data. 

The synchronization by the use of time information is more advantageous than 
that by the use of the position data and has a lower rate of failure. 

In the meantime, as described above, the standard MP3 file has a format 
comprising a header, a CRC, audio data and additional data. It can be considered that the 
20 caption information is included in the additional data of the standard MP3 file. However, 

since the compression rate varies depending on the amount of the audio data, it is 
therefore easier to add the caption information as a separate file format as in the first and 
second embodiments according to the present invention than to include the caption 
information in the additional data. Further, there are many advantages such as the 
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prevention of noise by the interruption occurring between the caption information and the 
audio information. 

FIG. 5 is a flow chart showing the method of reproducing the caption information 
in the caption MP3 file according to the second embodiment of the present invention. 

5 The file stored in the recording medium is opened; it is identified using the 

caption identification code (62) of the caption information (54) whether the caption 
information is in the file to be reproduced; the caption information is stored in the storage 
section (12) of the caption MP3 player if the caption information is included in the file 
and then the caption information is reproduced. When the reproduction of the caption 

10 information initiates, the caption information is initialized (step 80). At the step of 

initializing the caption information (80), it is determined whether to delete the caption 
displayed on the display device and to which file the caption data is attached. The 
reproduction terminates at the termination step (85). Otherwise, the playing time, which is 
reproduced in units of 1/1000 sec, is brought (step 86). This playing time has a value 

15 counted by the reproduction apparatus, for example, the control section (10) of the 

caption NT3 player shown in FIG. 1. The playing time is compared with the caption time 
(step 88). The caption time, as a caption display time (64a) of the caption data (58), is the 
time data value of a point of time at which the caption is displayed. The next caption time 
data is brought (step 90) and then the caption information is outputted (step 92). After 

20 returning to the termination- of- reproduction determining step 84 (step 94), steps 86-92 

are repeated. 

Although the present invention has been described with reference to the drawings, 
it is understood that this description is not to limit the invention to the embodiments 
25 shown in the drawings but simply to explain the invention. One skilled in the art will 

understand that various changes and modifications can be made from the embodiments 
disclosed in the specification. Therefore, the scope of the present invention should be 
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Abstract of the Disclosure 

A caption MP3 player having a caption display function of displaying caption 
information in synchronism with corresponding audio information when the audio 
information is outputted on stereo, a caption MP3 data format and a method of 
5 reproducing the caption MP3 data are provided. The caption MP3 player includes a 

storage section to which the audio information and the corresponding caption information 
are inputted and stored therein, a signal separation section for separating the audio 
information and the caption information provided from the storage section, a control 
section for controlling storage and output of the information through the storage section 
10 and controlling the audio and caption information separated by the signal separation 

section to be synchronized with each other, and a caption output section for outputting the 
caption information synchronized with the audio information that corresponds to the 
caption information and is outputted from the audio output section. 
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